Welcome back, you are watching the second video of a free course that will teach you everything you need to know about using Ollama and will help you become a pro with the technology. I’m releasing a new video in the course each week so keep coming back to learn more and more about it. In the previous video, we did a quick overview of how to get started with Ollama. In this video, we are going to go into more detail on installation. There is no way for me to gauge how much time you need to do any step, so it’s probably worth watching the video all the way thru without actually following along. Then try it out. If you run into any issues, come back to the video to see what you have missed. Pause, go back and speed forward are all options in the YouTube interface.

In the getting started video we saw that you had to go to ollama.com and in the middle of the page is a link to download it. So let’s look at the options. There are three options: macOS, Linux, and Windows. let’s start with Windows. I don’t actually have a windows system to use as all the computers in my house are Macs, so I will be working with an instance on Paperspace. Paperspace is part of Digital Ocean and after all my searching is the one reliable source of Windows machines in the cloud with named GPUs, meaning not just a fake name that Azure uses. This is a windows instance with a P6000 gpu that has 24gb vram and 32gb ram.

So once it finally starts up, you can go to ollama.com and start the download for the installer. Then run the installer. click through the buttons and its pretty easy. then you get this notification that ollama is started and then is set to run when you login. Now you may notice that I didn’t say anything about Nvidia or AMD drivers. If the drivers are all configured for your machine, then there is nothing else to do. Ollama will use the GPU if it’s a supported GPU. You can find the GPUs that are supported by going to this URL. If your GPU is on there, but you aren’t seeing ollama using the gpu, you can try either the course Discord or the Ollama Discord. The links to both of those are in the description below.

At this point you may be confused. Some folks expect a graphical UI to pop up. But the UI with ollama without installing anything else is at the command line. So we need to start by opening up either the terminal or powershell. Then you can run ollama run and the name of the model. I used phi3 in the last video because it’s nice and small. So ollama run phi3. And you should be plopped into the repl. Remember, last time I talked about the repl being a place you can go to ask a question and interactively get the answer. Now you can ask any questions you like just like you saw in the getting started video.

Of course there are always edge cases where the install doesn’t work. As I said before, try signing up to the two discords and you should get an answer pretty quickly.

Let’s move on to installing on Linux. For this I am using an instance I just created on Brev.dev. I love these guys. They make it so easy to create a Linux instance with any gpu you want. And no, they aren’t paying me to say that. There are just so many sketchy vendors out there so Brev is refreshing.

With Linux you download and run a script. Some folks find that a bit scary to do, especially if they don’t trust the source. So you can also review the script first or go to the manual install instructions. Most of the script just deals with drivers for video cards. The parts that are ollama specific copy the executable, create a user called ollama to run the executable and create the service to run in the background.

So run the script. Just like the windows version, if your drivers for your gpu are set then the install takes very little time. And then just like windows you can run ollama run and the model name. Now ask any question. There are some distributions of Linux that make things more difficult but if you are using them you already know this. Again, the discords are the best place to get help if you run into any issues.

With that done let’s move to the third platform which is macOS. It’s a universal app, so it will run on Apple Silicon or Intel Macs but there is a catch. It runs great on Apple Silicon. But Intel is more of a challenge. GPU support on those older Macs is non existant. The M1 came out about 4 years ago and is so superior to the Intel versions. I really doubt support will come for those older versions. But you can easily get a new Mac Mini with 16gb ram for about 800 which is pretty great. Installing Ollama on mac uses an installer that you download and run. But like the others it’s super quick and your done. Then open a terminal and run ollama run and a modelname. And boom. Ask your question.

There are a number of common next steps that folks want to deal with. One is installing a webui. I will have a number of videos on that in the future as part of this course. Another common need is to put models in a different directory from where it defaults to. I’ll have a video coming soon on where files go in Ollama, but if you want to figure out how to redirect models to a different directory on your machine, look into using environment variables in the ollama docs. Its not as easy as just setting up an environment variable in your login shell. Some folks will try to solve this by using symbolic links, but there are other issues that come up using that approach. The environment variables are the right way to go.

That’s really all there is to getting Ollama installed and up and running. Watch out for the next video in this course coming soon. Thanks for watching, Goodbye.